Localization: AI translation primitives#25705
Conversation
Generated by 🚫 Danger |
|
| App Name | Jetpack | |
| Configuration | Release-Alpha | |
| Build Number | 32947 | |
| Version | PR #25705 | |
| Bundle ID | com.jetpack.alpha | |
| Commit | f66c215 | |
| Installation URL | 41o4hkb6dv9u0 |
|
| App Name | WordPress | |
| Configuration | Release-Alpha | |
| Build Number | 32947 | |
| Version | PR #25705 | |
| Bundle ID | org.wordpress.alpha | |
| Commit | f66c215 | |
| Installation URL | 2a6mvva22pdi8 |
Reusable, unit-tested Ruby primitives for the AI translation tier of the localization pipeline — the service behind the `human ?? AI ?? English` floor whose AI stub was left open in #25688. Pure prompt-building and validation with the Anthropic SDK call injected, so the logic is testable without the gem or the network. Not wired into any lane yet. - TranslationValidator: format-specifier safety gate — a translation must preserve the source's placeholders (count and type; positional reordering allowed), or it is rejected and falls back to English. - Glossary: brand do-not-translate list plus per-locale terms and register. - AITranslator: single-string, per-key plural form-set (one consistent stem across CLDR forms), and batched string translation, with structured-output (output_config) enforcement. - AnthropicBatch: Message Batches submit/await/results/collect for bulk backfill. 50 unit tests, rubocop clean.
The pure-Ruby unit suites (TranslationValidator, Glossary, AnthropicBatch, AITranslator) weren't executed by any pipeline step — the "Unit Tests" jobs are the Xcode/XCTest suites, and rubocop (via Danger) only lints them. Add a lightweight Buildkite step that runs each fastlane/lanes/*_test.rb with plain ruby (stdlib minitest — no Xcode, no app build, no bundle). Runs unconditionally rather than behind should-skip-job.sh --job-type validation, which skips on tooling-only changes — i.e. exactly the PRs that touch these files.
28b37b5 to
7412850
Compare
The previous note advertised for_plural as a one-line swap to wire the live translation tier. That path routes each plural form through single-string translate, so it forfeits the cross-form consistency translate_plural exists to provide — the lemma drift PLURAL_OUTPUT warns about. Relabel for_plural as the per-cell fallback and point the live-tier wiring at translate_plural's form-set seam.
…f a translated value (#25721) * Localization: assert clean() preserves quotes that are part of a value A translation whose value is itself wrapped in quotation marks must keep them; only the model's cosmetic wrapping around a raw single-string reply should be stripped. Cover both structured paths (translate_plural, translate_all). * Localization: stop clean() stripping quotes that are part of a value clean() removes the cosmetic quotes a model wraps around a raw single-string reply. The plural and batch paths ran it on values already decoded by JSON.parse, so a value whose own content is quoted (e.g. "Reader") lost its quotes too. Run clean() only on the raw single-string reply in translate(); the JSON-decoded plural/batch values are whitespace-trimmed but never quote-stripped, since JSON.parse has already removed the structural quotes and anything left is content. Also covers the async collect_batch path (shares validated_batch) and curly-quoted values for free. Satisfies the tests added in d923c25. * Localization: cover curly quotes and the batch path in the quote-preservation tests Two more regression guards for the clean()-on-decoded-value fix: a curly/smart-quoted value (“Reader”) through translate_all, since clean() strips “ ” as well as straight quotes; and a quoted value through the async collect_batch path, which shares validated_batch with translate_all. Both fail against the pre-fix code, so a narrower fix — only un-stripping straight quotes, or only the sync path — cannot slip past. --------- Co-authored-by: Jeremy Massel <1123407+jkmassel@users.noreply.github.com>
oguzkocer
left a comment
There was a problem hiding this comment.
Looks good ![]()
I've left one nitpick comment which can possibly apply to a couple other function names, but it's a judgement call, so I'll leave the decision to you.
| end | ||
|
|
||
| # Map each numbered item to its validated translation by key; drop empty/placeholder-breaking ones. | ||
| def validated_batch(parsed, numbered) |
There was a problem hiding this comment.
At call sites, I read this function name as "The contents in this batch are already validated" which confused me. However, the actual implementation is "Take this batch and return the validated values" which means it's going to filter them.
Specifically, the 'validated' verb form misled me and for a minute I thought we weren't calling TranslationValidator.placeholders_match? for batches.
This could totally be a me problem, but I think an active version of the verb, or a more verbose version of the function name overall could improve readability at call sites.
There was a problem hiding this comment.
Renamed to select_valid_batch, which is Ruby-idiomatic. Also renamed validated_forms to select_valid_forms to match in f66c215
At call sites the validated_ prefix reads as an adjective — "the batch that's already been validated" — when both methods are in fact where batch and plural-set translations run the placeholder gate, returning only the passing subset. select_valid_batch / select_valid_forms make the filtering action plain where they're called. Pure rename of two private helpers; no behavior change.


Reusable Ruby primitives for the AI translation tier of the localization pipeline — the service behind the
human ?? AI ?? Englishfloor whose AI stub (ai_translate_plural→nil) was left open in #25688. All of it is pure prompt-building + validation with the Anthropic SDK call injected, so every line of logic is unit-testable without the gem or the network; the live SDK wiring is one thin factory.Nothing is wired into a lane yet — these are the building blocks, deliberately decoupled from the GlotPress / catalog plumbing that's still in flux.
Summary
TranslationValidator— the format-specifier safety gate. A machine translation must preserve the source's printf/NSString arguments exactly (count + type; positional%1$@may reorder, which is the whole point). A mismatch is rejected, so a broken translation falls back to English rather than shipping a crash in a locale no one on the team can read.Glossary— brand do-not-translate list (WordPress,Jetpack, …) plus per-locale preferred terms and a register note, rendered into the prompt. Pure data in; sourcing it (the WordPress.org per-locale glossaries / style guides) is pre-processing handed in later.AITranslator— three translation shapes:translate(one string),translate_plural(a whole CLDR form-set in one request, so the model keeps one stem across forms), andtranslate_all(batched regular strings). Structured outputs (output_configjson_schema) enforce the reply shape on the plural and batch paths.AnthropicBatch— the async Message Batches path for a bulk backfill (~50% cheaper):submit→await(poll) →results→collect_batch, plus all the SDK-shape glue, shared with the sync path so the request shape can't drift between them.Design notes
The SDK call is injected
AITranslatortakes acomplete:callable (and the batch path takes aclient), so prompt-building, validation, batching, and result-assembly are all exercised by unit tests with a canned reply.AITranslator.with_anthropic/AnthropicBatch.clientbuild the live instances (default modelclaude-opus-4-8). The two live-API bugs we hit — acustom_idthat didn't match^[a-zA-Z0-9_-]{1,64}$, andresults_streamingyielding raw JSONL strings rather than typed objects — were both invisible to a permissive fake client and only caught by real calls. The SDK seams are now live-verified, not just fake-tested.The placeholder gate is a hard floor
Every machine cell — single string, plural form, or batch entry — passes through
TranslationValidatorbefore it's returned. This is the same invariant the catalogneeds_reviewmachinery already assumes: the AI tier can only ever produce a safe translation ornil.Plural consistency
Translating each CLDR category independently let the model drift between synonyms across forms (Polish
słowo→wyrazy→słów).translate_pluralsends the whole form-set in one request and instructs one consistent stem; verified it now yieldssłowo / słowa / słów.Verified live
Against the real API (fr/de/ja/pl): verb/adjective disambiguated from the dev comment (
Suivre/Suivi,Folgen/Gefolgt), brand terms kept verbatim, German informal register (Dein), French space-before-?, plural stems consistent, and a full Batch round-trip (submit → await → collect) returningSuivre/%1$@ vuesmapped back to keys and grouped by locale. Placeholders were preserved throughout — the gate never had to fire on real output.Not in this PR (deliberate)
download_localized_pluralsstill calls thenilstub; switching it toAITranslator#for_plural(and atranslate_allpass over the regular catalog) is a separate change..strings/.xcstringsreader — these consume a{key, source, comment}array and plural form-sets; building that from the real catalog (plus any pre-processing) is upstream of here.Glossaryis data-in; pulling the WordPress.org per-locale glossaries / style guides into it comes later.await).Test plan
All checks are pure Ruby — stdlib minitest, no bundle, no network:
ruby fastlane/lanes/translation_validator_test.rb— 9ruby fastlane/lanes/translation_glossary_test.rb— 5ruby fastlane/lanes/anthropic_batch_test.rb— 6ruby fastlane/lanes/ai_translator_test.rb— 30rubocopclean on all eight filesANTHROPIC_API_KEY+bundle install):ruby fastlane/lanes/ai_translator.rb fr "You have %1$d new posts" "Notification. %1$d is the count."Related